Mining User Interests from Personal Photos

Personal photos are enjoying explosive growth with the popularity of photo-taking devices and social media. The vast amount of online photos largely exhibit users’ interests, emotion and opinions. Mining user interests from personal photos can boost a number of utilities, such as advertising, interest based community detection and photo recommendation. In this paper, we study the problem of user interests mining from personal photos. We propose a User Image Latent Space Model to jointly model user interests and image contents. User interests are modeled as latent factors and each user is assumed to have a distribution over them. By inferring the latent factors and users’ distributions, we can discover what the users are interested in. We model image contents with a four-level hierarchical structure where the layers correspond to themes, semantic regions, visual words and pixels respectively. Users’ latent interests are embedded in the theme layer. Given image contents, users’ interests can be discovered by doing posterior inference. We use variational inference to approximate the posteriors of latent variables and learn model parameters. Experiments on 180K Flickr photos demonstrate the effectiveness of our model. Introduction With the prosperity of photo-taking devices such as digital cameras and smart phones, people habitually take photos to record interesting stuff and memorable events in their daily life. Everyday, millions of photos are uploaded to photo sharing social networks, like Flickr, Pinterest and Instagram. Personal photos reveal people’s interests explicitly or implicitly. For instance, people loving pets tend to shoot a lot of dog and cat images and share them on social media. People enjoying food frequently populate their online albums with various food images. Figure 1(a) shows photos of four Flickr users. Browsing these photos, we can easily figure out that the first user likes cars, the second user is fond of flowers, the third user loves football and the fourth user enjoys food. A picture is worth a thousand words. Compared with texts, images are more natural to express users’ interests and emotion. Mining users’ interests from their personal photos Copyright c © 2015, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Car Tiger Tree Car Tiger Tree Car Tiger Tree